1. Field of the Invention
The present invention relates to an image processing apparatus, an image processing method, and a program, and more particularly, to an image processing apparatus, an image processing method, and a program that are capable of extracting a foreground object image from an input image and a binary mask image (binary image) without causing color blur.
2. Description of the Related Art
Alpha matting is a general technique for cutting out, as a foreground object image, a certain region of a digital image such as a picture.
Cutting out a foreground object image by multiplying an alpha mask image generated by the alpha matting technique and an input image together and combining the foreground object image with a new background image is considered.
Here, an alpha mask image is represented using values indicating the levels of transparency of individual pixels in a foreground object image (hereinafter, also referred to as α values). That is, an alpha mask image is a transparency level image including pixels representing the levels of transparency in a foreground object image. In an alpha mask image, each pixel value is represented using the transparency α of the color arrangement in a foreground object image. For example, 100-percent transparency is represented by α=1, 0-percent transparency is represented by α=0, and 50-percent transparency (halftone) is represented by α=0.5. Thus, an image obtained by multiplying pixel values in a foreground object image by the transparencies α of corresponding pixels in an alpha mask image is extracted as a foreground object image.
Here, pixels around the edge of the foreground object image may retain the color of a background image of the input image.
This occurs because an input image originally includes an edge portion of a foreground object image in which the color of the foreground object image is mixed with the color of a background image and the color-mixed edge portion becomes apparent as color blur when the foreground object image is combined with a new background image. Thus, in order to achieve more natural combining of a cut-out foreground object image, it is necessary not only to obtain an alpha mask image but also to obtain the foreground object image by calculating the true color of the foreground object image, so that the foreground object image and the alpha mask image can be multiplied together and used for the combining.
For example, the methods described below are foreground color estimation methods used in the alpha matting technique described above.
A first method is a method for minimizing, in accordance with an alpha mask image obtained using alpha-matting and an input image, differences between pixel values of the input image and pixel values obtained from an estimated foreground image, an estimated background image, and the alpha mask image and for minimizing a first-order differential difference (see A. Levin, D. Lischinski, and Y. Weiss, “A Closed Form Solution to Natural Image Matting”, 2006 Conference on Computer Vision and Pattern Recognition (CVPR 2006), June 2006, pp. 61-68). In this method, on the basis of the prediction that an alpha value, a foreground color, or a background color greatly changes at a pixel position at which the pixel value greatly changes, a smooth foreground image can be obtained.
A second method is a method for producing simultaneous linear equations under the constraint of a difference in an α value from an alpha mask image obtained using alpha matting and an input image, complementing pixel values of the input image, and estimating the foreground color (see Jue Wang, Maneesh Agrawala, and Michael F. Cohen, “Soft Scissors: An Interactive Tool for Realtime High Quality Matting”, ACM Transactions on Graphics, 26(3), July 2007, pp. 9:1-9:6). The second method is effective because only an alpha difference value is put under constraint.
A third method is a method for, when an alpha mask image is calculated using alpha matting, employing a tentative foreground color obtained in a transitive manner as an estimated foreground color (see Yung-Yu Chuang, Brian Curless, David H. Salesin, and Richard Szeliski, “A Bayesian Approach to Digital Matting”, In Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR 2001), Vol. II, 264-271, December 2001). In the third method, an alpha value is calculated by sampling a plurality of candidates of foreground color and background color from pixels surrounding a pixel at a position to be calculated.
A fourth method is a method for manually removing color blur by using image processing software on an alpha mask image obtained using alpha-matting and an input image. The fourth method is provided as a “defringe function” (see http://help.adobe.com/ja_JP/Photoshop/10.0/help.html?content=WSfd1234e1c4b69f30ea53e41001031ab64-76de.html).